In traditional manufacturing environments, where an assembly line builds some quantity of physical things, product quality is predicated in part on inspection and the ability to actually touch, see, feel and otherwise examine the final product. Many other quality control techniques are based on this fundamental ability.
In software development much attention has been given to the many elements that contribute to the creation and deployment of a software system. Such elements are all part of a “software assembly line” that makes up the processes that software developers use when developing and deploying a software system. But what has long been ignored is the fact that software products are themselves assembly lines and an important aspect is whether an application performs correctly for the user, in their environment, with their specific inputs, in the exact way in which they use the application. Every time the user runs the application, they are using the software assembly line to manufacture another final product through a specific run of the application. Although the software may work perfectly in some situations, it can fail completely in others, just as some products that come off a traditional assembly line may be within specifications while others may be completely or partially unusable.
In order to enable inspection on the application assembly line, which provides the basis for immense software quality improvements, applications must have an attribute of reproducibility. In other words, applications must have an ability to recreate any and all specific runs of the application. Unfortunately, with conventional software, a system evaluator must re-enter the user data in the proper sequence to debug a failure. One measure of testability for software is how easily a specific run of the software system can be reproduced.
The software developers have adopted various methodologies to aid in the software product development process. One principle of many of these techniques, requirements-based design and code reuse, has had an enormous impact especially on the design and implementation phases of development. However, when it comes time to test and release a software product, there often is a disconnect between the product's functional requirements and the product itself because there are no commercially available software testing products that deliver any real measure of testability for software products. Software companies must rely largely on the sheer quality of their programming talent to verify the functional integrity of any final product. Thus, software companies are understandably cautious when hiring, because the software development process currently is so sensitive to the quality and skill levels of each individual developer on the team.
Feedback is one attribute that enables hardware engineers to easily control and stabilize the system, in many classes of hardware systems (amplifiers, control systems, etc.). In software systems, testability, which enables feedback, is largely absent. As software applications grow in complexity, both in size and in their interconnections with other applications, the need for strong feedback in the system also grows.
In software systems, testability is one measure of how easily an application can be made to reproduce any specific run of the software system, or how easily feedback can be enabled in the system. FIG. 1 depicts one example of a current software test system. Although commercial black-box 28 software test products such as this are available, they all suffer from the fact that while they can reproduce some runs of an application 22, typically with a considerable amount of effort, they do not actually increase the testability of the application 22 at all. If a real user experiences a failure in the application, it doesn't matter how many canned test scripts have been created for the application. If the application is not testable it simply will not be able to recreate the conditions and actions that make up the specific run of the application 22 that failed for the user. Additionally, such a system 20 utilizing a black-box 28 test method requires a test engineer to learn a new scripting language 24 as well as undertake continuous maintenance 26 to insure the black-box 28 is functioning properly.
What is needed is an application that enables feedback with near insignificant effort by the application programmer to allow the application programmer to easily control and stabilize the software system. What is also needed is a software testing technology that enables complete Push-Button Reproducibiity™ for any software system, even those with asynchronous properties such as Graphical User Interfaces (GUI) or distributed processes, with an insignificant amount of effort by the application programmer.
Additionally, a “white box” testing solution is needed that enables zero-effort Push-Button Reproducibility™ and testability for any software system. In effect, what is needed is an application that adds feedback to the software development process, bridging the gap between the promise of requirements-based design and the realities of software testing and maintenance. What is needed is an application to make a software system easy to test and verify, allowing companies to reap the rewards of code reuse without worrying if some change in one section of code is going to cause failures when that code is used in another part of the program.
Solving these problems would allow companies to stop relying on the sheer magnitude of their programmers' talent for software verification, so that even new college graduates can easily contribute to the software development process. While it may take a recent graduate longer than a more experienced engineer, the final product will be of the same quality in that testing and verification is no longer a function of the programmer's skill level.